Adaptive Margin Support Vector Machines
نویسنده
چکیده
In this chapter we present a new learning algorithm, Leave{One{Out (LOO{) SVMs and its generalization Adaptive Margin (AM{) SVMs, inspired by a recent upper bound on the leave{one{out error proved for kernel classiiers by Jaakkola and Haussler. The new approach minimizes the expression given by the bound in an attempt to minimize the leave{one{out error. This gives a convex optimization problem which constructs a sparse linear classiier in feature space using the kernel technique. As such the algorithm possesses many of the same properties as SVMs and Linear Programming (LP{) SVMs. These former techniques are based on the minimization of a regularized margin loss, where the margin is treated equivalently for each training pattern. We propose a minimization problem such that adaptive margins for each training pattern are utilized. Furthermore, we give bounds on the generalization error of the approach which justiies its robustness against outliers. We show experimentally that the generalization error of AM{SVMs is comparable to SVMs and LP{SVMs on benchmark datasets from the UCI repository.
منابع مشابه
A QUADRATIC MARGIN-BASED MODEL FOR WEIGHTING FUZZY CLASSIFICATION RULES INSPIRED BY SUPPORT VECTOR MACHINES
Recently, tuning the weights of the rules in Fuzzy Rule-Base Classification Systems is researched in order to improve the accuracy of classification. In this paper, a margin-based optimization model, inspired by Support Vector Machine classifiers, is proposed to compute these fuzzy rule weights. This approach not only considers both accuracy and generalization criteria in a single objective fu...
متن کاملIdentification and Adaptive Position and Speed Control of Permanent Magnet DC Motor with Dead Zone Characteristics Based on Support Vector Machines
In this paper a new type of neural networks known as Least Squares Support Vector Machines which gained a huge fame during the recent years for identification of nonlinear systems has been used to identify DC motor with nonlinear dead zone characteristics. The identified system after linearization in each time span, in an online manner provide the model data for Model Predictive Controller of p...
متن کاملAdaptive Margin Support Vector Machines for Classi
In this paper we propose a new learning algorithm for classiication learning based on the Support Vector Machine (SVM) approach. Existing approaches for constructing SVMs 12] are based on minimization of a regularized margin loss where the margin is treated equivalently for each training pattern. We propose a reformulation of the minimization problem such that adaptive margins for each training...
متن کاملAdaptive Nearest Neighbor Classiication Using Support Vector Machines
The nearest neighbor technique is a simple and appealing method to address classiication problems. It relies on the assumption of locally constant class conditional probabilities. This assumption becomes invalid in high dimensions with a nite number of examples due to the curse of dimensionality. We propose a technique that computes a locally exible metric by means of Support Vector Machines (S...
متن کاملAdaptive Nearest Neighbor Classification Using Support Vector Machines
The nearest neighbor technique is a simple and appealing method to address classification problems. It relies on the assumption of locally constant class conditional probabilities. This assumption becomes invalid in high dimensions with a finite number of examples due to the curse of dimensionality. We propose a technique that computes a locally flexible metric by means of Support Vector Machin...
متن کامل